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MUSICAL PERFORMANCE SELF-TRAINING APPARATUS 

BACKGROUND OF THE INVENTION 

Field of the Invention 

The present invention relates to a musical performance 
self-training apparatus, and more specifically, relates to a 
musical performance self-training apparatus that can enable 
efficient lesson by designating a lesson part of music to be 
performed, corresponding to the skill level of a player in the 
musical performance for a given music. 

The present invention also relates to a musical 
performance self-training apparatus that can avoid such a 
situation that the training becomes boring, by having a mode 
in which a player him/herself can determine the training 
progress and a mode in which the progress is instructed 
automatically. 

Moreover, the present invention includes a musical 
performance judgment apparatus that can specifically evaluate 
if musical performance has been done as instructed by a 
performance instruction based on performance data prepared in 
advance. 

Description of the Related Art 

There is known an apparatus which plays performance data 
stored in a recording medium by a personal computer, and 
displays an image representing sheet of music sequentially on 
a screen, to thereby give a performance instruction. For 
example, in Japanese Patent Application Laid-Open No. Hei 



9-305171, there is disclosed an apparatus which gives a 
performance instruction in a manner of a music box in which keys 
are instructed sequentially by graphical bars. More 
specif ically, in this apparatus , a figure of a keyboard is 
5 displayed on a screen, and a scroll bar having a length 
corresponding to the duration of key depression is also 
displayed corresponding to each key on the displayed keyboard, 
and this scroll bar is scrolled so that the scroll bar approaches 
the keyboard figure, as the auto-playing data is played. 

10 According to this conventional performance instruction 

apparatus, the player can recognize the duration of key 
depression intuitively, and a key to be depressed subsequent 
to the key being depressed now can be known in advance. 
Therefore, even a player who cannot read the musical score can 

15 play smoothly, different from the apparatus which indicates 
performance with displaying the musical score. 

However, with the conventional performance instruction 
apparatus, since the performance data is only played 
sequentially from the beginning which is monotonous, efficient 

20 training cannot be conducted and the player become bored. For 
example, it is quite difficult for a player, who is training 
for given music for the first time, to play from the top of the 
music to the very end all the way through. Moreover, even if 
the player interrupts on the way to return to the beginning, 

25 or return to a desired training part, it is troublesome and 
time-consuming to search the part to be played. 

In a music the same phrase appears in a plurality of places , 



even when this phrase is mastered, when the player has the 
training all the way through, he/she has to play this phrase 
many times, and hence lesson may becomes boring. It is not 
efficient and not desirable to have repetitive training for the 
5 phrase mastered to a certain degree, from a viewpoint of 
mastering the entire music as soon as possible. 

Therefore, a system in which the skill level of the entire 
music can be efficiently enhanced is desired, by avoiding 
repetition of training for the mastered phrase. 

10 Moreover, even if there is a part which the player wants 

to train selectively, with the apparatus which advances 
training so that the entire music can be mastered at a certain 
skill level, selection of the training part or training progress 
against his/her will may be forced on the player. 

15 In order to solve the above problem , a mode can be 

considered in which the player can freely determine the training 
progress to play, but if this mode is for freely playing far 
apart from the automatic mode in which the progress is 
instructed automatically, the relation between the automatic 

20 mode and this mode becomes weak, and hence consistent training 
cannot be performed. There is also a demand for a musical 
performance self -training apparatus, which keeps freedom in 
selecting progress by the player, while maintaining the 
training function by the automatic mode. 

25 Furthermore, even if a player who cannot read the musical 

score can play certain music by an apparatus which displays a 
performance instruction in a manner of a music box, it is not 
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applicable to a case in which another music is played, since 
the player is not able to become read the score. In other words , 
it is difficult to play without the performance instruction in 
the manner of the music box at all times. 
5 Therefore , there is a demand for an apparatus which makes 

it possible not only to play particular music but also to finally 
play any music by reading a score, by appropriately adopting 
training by the score, while having training according to the 
performance instruction displayed in the manner of the music 
10 box . 

Since the scroll bar is scrolled forward when a player 
has played at a right pitch, it stands by until the player plays 
at a right pitch, without proceeding to the next performance 
instruction. By this method, training can be steadily advanced, 
15 while confirming whether each note is played at the right pitch. 
However, according to this instruction method, the performance 
is standardized, and variations in performance are not allowed. 
Therefore, it is insufficient in view of enjoying the musical 
performance. 

20 Therefore, a musical performance judgment apparatus, 

which is set such that even if the music performance is not 
conducted as instructed by the performance data, the music is 
advanced so long as the musical performance is within a range 
of a predetermined pitch, has been proposed (Japanese Patent 

25 Application Laid-Open No. 2002-175072). In this publication, 
it is also disclosed that players can learn performance 
techniques while enjoying the music, by allowing liberal 
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judgment for the performance of music having high degree of 
difficulty or by adjusting the judgment method and the judgment 
standard in the same performance data, depending on the age and 
ability. 

As described above, it has been heretofore considered 
that a player has training while confirming the played tone one 
by one, to learn the key-on sequence, and then gives a musical 
performance along to the accompaniment and metronome sound, 
thereby the player can finally give a natural musical 
performance with good tempo. 

However, with respect to the rudimentary training in 
which a player gives a musical performance while confirming the 
played note one by one, and the advanced training in which the 
player gives a musical performance along to the accompaniment 
and metronome sound, a single-level performance instruction 
cannot correspond to these various level of trainings, since 
the difference in degree of difficulty between these trainings 
is too large. 

If the ability of a trainee is finely judged, to adjust 
the judgment method and the judgment standard for the training 
highly accurately, the trainee can give a musical performance 
reasonably, and improvement can be expected. However, the 
judgment method and the judgment standard have heretofore been 
adjusted only according to whether the pitch is as instructed. 

Therefore, there is a demand for a system, which can 
multilaterally judge the performance result of a trainee to 
recognize the ability, and determine the judgment method and 



20 



25 



the judgment standard based on the ability. 

There is also proposed an apparatus which can output 
performance evaluation during performance or after finishing 
the performance (Japanese Patent Application Laid-Open No. 
5 2001-242863). Moreover, there is known an apparatus in which 
when the music has not been played as instructed, for example, 
when a key of the right note has not been pressed, the next 
performance instruction is not given, nor the performance is 
not evaluated. 

10 However, if the performance instruction is stopped when 

a player has not played correctly, a problem may occur. When 
the player has noticed a miss-touch, he/she can replay again 
immediately, but if the player has noticed a miss-touch after 
having played for a while, he/she must return to the beginning 
to give a performance again, in this case, since the evaluation 
after having stopped the performance instruction is not given, 
the player cannot know the performance evaluation, and must 
return to the beginning to give a performance again, without 
being convinced. 

With such a performance instruction apparatus, the player 
pays attention only to the performance instruction, and cannot 
enjoy the performance. Trill, tremolo and the like are ways 
of rendition finely repeating a plurality of notes, but the 
number of repetition should be essentially left to the 
sensibility of the player, with the conventional performance 
instruction apparatus, however, if the number of repetition is 
not performed as instructed, the performance instruction is not 



15 



6 



advanced forward. Therefore, there is a demand for an apparatus 
which enables performance by giving much weight to the 
performance flow, and which can express the sensibility of the 
player. 

5 In a conventional performance instruction apparatus, a 

player can only get information, which can be read from notes 
and a clef described on a musical score, and information 
indicating the performance timing. However, higher 
performance technique such as fingering cannot be understood 
10 from the musical score. Moreover, a beginner may not be able 
to understand the meaning of the sign written on the musical 
score. 

As described above, in order to actually give a good 
performance, there are matters which a player should know other 

15 than the information expressed by the musical score, but in the 
conventional performance instruction apparatus, the 
information cannot be informed. Therefore, there is a demand 
for means with which the player can search the information, 
which is not written in the musical score, as required. 

20 SUMMARY OF THE INVENTION 

It is the first object of the present invention to provide 
a musical performance self -training apparatus, which makes it 
possible to repetitively train for a part having not mastered 
yet, to thereby efficiently master the entire music. 

25 It is the second object of the present invention to provide 

a musical performance self-training apparatus, which can change 
the training progress by the player him/herself, while 
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maintaining the function capable of instructing progress 
automatically. 

It is the third object of the present invention to provide 
a musical performance self -training apparatus, which aims at 
5 being able to read the score gradually , while playing according 
to the performance instruction. 

It is the fourth object of the present invention to provide 
a musical performance judgment apparatus , which can 
multilaterally judge the performance result of a trainee to 
10 judge the ability of the trainee highly accurately. 

It is the fifth object of the present invention to provide 
a musical performance self -training apparatus , which can give 
the performance instruction and evaluation of the performance 
result, without impairing the music flow and the sensibility 
15 of the player. 

It is the sixth object of the present invention to provide 
a musical performance self-training apparatus , which can search 
information other than the information written in the musical 
score, according to need, during the training. 
20 In order to accomplish the above objections, the present 

invention is constructed as follows. 

The apparatus of the present invention comprises a unit 
designating means for designating a unit from the plural units, 
the units constitute a music to be performed and each unit 
25 includes a predetermined size of musical tone information, and 
a performance instruction information generating means for 
generating the performance instruction information based on the 



musical tone information of the unit which is designated by the 
unit designating means. 

The apparatus of the present invention further comprises 
a lesson menu generating means for generating an image 
5 information of a lesson menu which has a score of the music to 
be played and the units corresponding to the score, then output 
the image information to the display means , wherein the unit 
designating means designates a unit automatically, which unit 
corresponding to the score of the performance instruction 

10 information to be displayed next. 

In the present invention, the lesson menu having the 
units which are provided to include wider range of musical tone 
information, as the skill level become higher, are displayed 
according to the skill level corresponding to the score, and 

15 the unit designating means designates a unit to be played at 
the earliest timing, in which performance thereof has not 
reached a predetermined acceptable standard, of the units on 
the lesson menu, as a unit of which performance instruction is 
to be displayed next, and when performance of all units in the 

20 same skill level has reached the acceptable standard, the unit 
of which performance instruction is to be displayed next is 
selected from units in the upper ranking skill level than the 
current skill level and designated. 

Accordingly, the player can confirm the music to be 

25 played by the lesson menu in which the musical score is included. 

A unit of the displayed music in the lesson menu is 
designated in order from the earliest one timewise, and the 



9 



performance instruction information included in the unit is 
displayed. A unit reached to an acceptable standard is not 
designated any longer, and the next earlier unit timewise is 
then designated. 

If the performance related to all units of the same skill 
level reach the acceptable standard, the player cannot advance 
to the upper level training, and hence the player can be reliably 
master the music. If the player can performance related to the 
units of certain level collectively, the player can advance to 
the lesson of the units in the upper skill level including these 
units . 

In the present invention, the unit designating means 
being constructed so as to be able to operate selectively in 
an automatic mode or manual mode. And, the automatic mode or 
manual mode is selected by operating the mode select switch 
displayed in the lesson menu on the display apparatus. 

Accordingly, the mode of designating unit is not limited 
to the automatic mode, that is, the player can designate the 
unit according to the lesson progression, and hence the lesson 
progression can be advanced or delayed, optionally. 

The present invention further comprises the performance 
judgment apparatus, wherein the performance judgment apparatus 
comprises a performance result storage means for storing 
information relating to performance timing as a result of 
musical performance, a judgment means for comparing the 
information relating to the performance timing with the 
information relating to the performance timing included in the 
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performance instruction and detecting the difference between 
these, and an evaluation means for evaluating the performance , 
corresponding to the difference detected by the judgment means . 

In the performance result storage means a plurality of 
items are stored in the performance result storage means for 
each played note, as the information relating to the performance 
timing, and the evaluation means extracts the items evaluated 
best of the plurality of items with respect to all played notes, 
to evaluate the musical performance according to the mean value 
thereof. 

In the present invention, the performance evaluation is 
graphically displayed for each of the information 
comprehens ively . 

Accordingly, the information relating to the performance 
timing is compared between the actual performance and the 
performance instruction to evaluate the performance. Therefore, 
the rhythmic sense of the player can be judged and the accuracy 
of performance can be judged, different from the evaluation by 
simply judging whether the pitch agrees with the pitch in the 
performance instruction. Particularly, the musical 
performance is evaluated in a plurality of evaluation items 
judged based on the information relating to the performance 
timing, and based on all tones through the performance. As a 
result, detailed judgment of ability becomes possible. 

Since a plurality of items is displayed comprehensively, 
the strong point and weak point of the player can be easily 
judged. 
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The apparatus of the present invention comprises a score 
view mode display means for displaying performance instruction 
information in which performance timing information is added 
to the note information arranged on a staff, a piano roll view 
5 mode display means which associates a mark having a length 
corresponding to the length of the note with a keyboard figure 
and displays the performance timing with a distance between the 
mark and the keyboard figure , and a changeover means for 
optionally selecting either the score view mode display means 

10 or the piano roll view mode display means. 

The display means of each mode is constructed so as to 
display the evaluation together with the performance 
instruction information, and is constructed so as to add a check 
to the note information or the mark of the note which has not 

15 been played as instructed, and display an annotation therefore 
together. 

Since the performance instruction can be displayed in 
an optional mode of the score view mode or the piano roll view 
mode, the performance instruction can be displayed in a display 
20 mode which a player desires. 

Moreover, the note which could not be played as 
instructed becomes clear, and a point different from the 
instruction can be specifically displayed by the annotation 
therefore. 

25 The apparatus of the present invention comprises a played 

note storage means for storing played notes, a means for 
suspending the progress of performance instruction when a note 
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is not played at a pitch as instructed by the performance 
instruction, a performance data row detection means for 
searching a performance data row whose array coincides with a 
plurality of played notes including at least the newest played 
5 notes stored in the played note storage means , from the 
performance data, and a control means for resuming the progress 
of performance instruction from the row subsequent to the 
performance data row, when the coincident performance data row 
is searched. 

10 The played note storage means stores a predetermined 

number of played notes after the played note, which has not been 
played at a pitch as instructed by the performance instruction. 

The played notes are stored in a buffer. When the note 
is not played at the instructed pitch, the performance 

15 instruction is suspended, and the performance data row, which 
agrees with the played note row stored in the buffer in view 
of the array, is detected. The performance instruction is then 
started from the row subsequent to the performance data row. 
In other words, the played notes are stored, and when a note 

20 has not been played at the instructed pitch, a pattern agreeing 
with the stored played notes in view of the array is searched 
in the performance data. In this manner, when a player has made 
a miss-touch or played an optional music part according to 
his/her preference, the key-on instruction is suspended once, 

25 but the performance position is immediately detected to resume 
the key-on instruction. 

When notes are played (in the case of keyed instrument, 
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key-on) at the pitch instructed by the performance instruction 
(in the case of keyed instrument , key-on instruction) , the 
performance instruction is given for the next score information. 
On the other hand, when the notes are not played as instructed, 
5 that is, when the notes are not played as instructed due to a 
miss-touch or due to a performance technique by intention, the 
played notes after the played note information are stored in 
the buffer for the predetermined number. The performance data 
corresponding to the played notes stored in the buffer is 

10 detected, and the key-on instruction is restarted from the note 
subsequent to the performance data. Therefore, the 
performance instruction catches up with the actual player's 
performance at a point in time when the corresponding 
performance data is detected. According to this feature, the 

15 area of the buffer can be reduced. 

The apparatus of the present invention comprises a means 
for displaying an advice icon, which informs the existence of 
advice information for musical performance, together with the 
performance instruction information, and a means for 

20 searching and displaying the advice information relating to the 
performance instruction information at a position where the 
advice icon is displayed, in response to the operation of the 
advice icon. 

The advice information is searched through the internet. 
25 When a player finds an advice icon, while reading the 

displayed musical score and playing, the player can operate the 
icon. Therefore, if there is anything which the player wants 



to know relating to the performance by the displayed musical 
score during the performance , he/she can operate the icon to 
search the advice information such as the playing technique. 
Particularly, since the advice information can be 
5 obtained and displayed through the Internet, it is not necessary 
to prepare various kinds of advice information beforehand in 
a memory of the musical performance self-training apparatus. 
BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a block diagram showing the main function of 
10 a musical performance self-training apparatus according to the 
first embodiment of the present invention; 

Fig. 2 is a system block diagram of the musical performance 
self-training apparatus according to the first embodiment of 
the present invention; 
15 Fig. 3 is a diagram showing a display example of a lesson 

menu; 

Fig. 4 is a flowchart showing the operation of the musical 
performance self -training apparatus; 

Fig. 5 is a flowchart showing unit designation 
20 processing; 

Fig. 6 is a diagram showing a display example of a key 
depression instruction; 

Fig. 7 is a diagram showing another display example of 
a lesson menu; 

25 Fig. 8 is a diagram showing an example of a lesson menu 

in which unit setting is inappropriate; 

Fig. 9 is a diagram showing an example of a lesson menu 

15 



in which unit setting is reset appropriately; 

Fig, 10 is a diagram showing an example of music in which 
the unit setting is to be changed; 

Fig. 11 is a flowchart showing the operation of the musical 
5 performance self-training apparatus according to a second 
embodiment; 

Fig. 12 is a block diagram showing the main function of 
a musical performance self-training apparatus according to a 
second embodiment of the present invention; 
10 Fig. 13 is a diagram showing another display example of 

the lesson menu; 

Fig. 14 is a diagram showing an example of key depression 
instruction in a score view mode; 

Fig. 15 is a diagram showing an example of key depression 
15 instruction in a piano roll view mode; 

Fig. 16 is a diagram showing an example of key depression 
instruction in the score view mode , including result display; 

Fig. 17 is a diagram showing an example of key depression 
instruction in the piano roll view mode, including result 
20 display; 

Fig. 18 is a diagram showing an example of key depression 
instruction in a superimpose mode; 

Fig. 19 is a diagram showing another example of key 
depression instruction in the superimpose mode; 
25 Fig. 20 is a diagram showing an example of main 

configuration of performance data; 

Fig. 21 is a block diagram showing the main function of 
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a musical performance self-training apparatus according to the 
third embodiment of the present invention; 

Fig* 22 is a flowchart showing the operation of a musical 
performance self-training apparatus according to the fourth 
5 embodiment of the present invention; 

Fig. 23 is a diagram showing one example of performance 

data; 

Fig. 24 is a diagram showing one example of the performance 
result stored in a memory; 
10 Fig. 25 is a diagram showing one example of 

sub-evaluation, which becomes the grounds for the comprehensive 
evaluation; 

Fig. 26 is a diagram showing one example of the 
comprehensive evaluation based on the sub-evaluation; 
15 Fig. 27 is a diagram showing one example of a displayed 

image, on which a graph showing the comprehensive evaluation 
is displayed on a key depression instruction display; 

Fig. 28 is a block diagram showing the main function of 
the a musical performance self-training apparatus according to 
20 the fourth embodiment of the present invention 

Fig. 29 is a flowchart showing input processing of key 
depression tones; 

Fig. 30 is a diagram showing an example of data structure 
in an evaluation storage area; 
25 Fig. 31 is a diagram showing an example of data structure 

in a played note storage area; 

Fig. 32 is a block diagram showing the main function of 
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a musical performance self-training apparatus according to the 
fifth embodiment of the present invention; 

Fig* 33 is a main flowchart in the sixth embodiment; 

Fig. 34 is a flowchart showing the input processing of 
5 performance result; 

Fig, 35 is a diagram showing a structural example of the 
played note storage area (buffer); and 

Fig. 36 is a main functional block diagram showing the 
main function of a musical performance self-training apparatus 
10 of the sixth embodiment; 

Fig. 37 is a diagram showing a display example of a key 
depression instruction; 

Fig. 38 is a flowchart showing the operation of the musical 
performance self -training apparatus; 
15 Fig. 39 is a diagram showing an example of advice 

information search data; 

Fig. 4 OA is a diagram showing a display example of key 
depression instruction information; Fig 4 OB is a diagram 
showing a display example of the searched advice information; 
20 Fig. 41A is a diagram showing another display example of 

the key depression instruction information; Fig. 4 IB is a 
diagram showing another display example of the searched advice 
information; and 

Fig. 42 is a block diagram showing the main function of 
25 a musical performance self-training apparatus according to one 
embodiment of the present invention. 
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DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

The present invention will be described in detail, with 
reference to the drawings. Fig. 2 is a block diagram showing 
the constituent of the musical performance self -training 
5 apparatus according to one embodiment of the present invention. 
In the Fig. 2, a personal computer 1, that is, a PC comprises 
a PC body 11, a keyboard 12 and a mouse 13 as an input unit, 
and a display 14 as an output unit. For the PC body 11, one 
having a known constituent having a hard disk, ROM, RAM and the 

10 like can be used. It is desired that the PC body 11 comprise 
an interface that can input and output an MIDI (musical 
instrument digital interface) signal. 

A keyboard instrument 2 comprises a keyboard 21 and a sound 
system 22. The keyboard instrument 2 comprises a detection 

15 circuit 23 connected to the keyboard 21 to detect key depression 
and key release information on the keyboard 21, and a tone 
generator 24 for generating tone corresponding to the key-on 
or key-off information. When the detection circuit 23 detects 
key depression or key release on the keyboard 21, the tone 

20 generator 24 generates the tone, based on the depressed key 
number (key number) , the key-on length, the key-on velocity and 
the like, and outputs (produces ) musical sound through the sound 
system 22. It is desired that the keyboard instrument 2 also 
comprise an MIDI. The overall operation of the keyboard 

25 instrument 2 is controlled by a microcomputer (not shown). 

The PC body 11 and the keyboard instrument 2 are connected 
via an interface such as the MIDI (not shown), so that a signal 



can be transferred between these. 

In the musical performance self-training apparatus, 
music information for training can be externally supplied, or 
one which is stored in a known storage medium such as a hard 
5 disk in the PC body 11 beforehand may be used. The music for 
training is prepared as performance data such as note number 
provided in the MIDI standard, key-on time, key-off time, 
velocity, and tempo data. A construction example of the 
performance data is shown in Fig. 20. The performance data may 

10 include data other than the above-described data, but it is not 
the main part of the present invention, and hence the 
description thereof is omitted. 

The selected music information can be played not only from 
the beginning, but may be played halfway of the music. The 

15 training part may be automatically designated, or may be 
designated manually by the player him/herself. The training 
part can be designated for each predetermined range (referred 
to as a unit). The unit comprises notes which are included in 
one or a plurality of bars, and as the degree of skill level, 

20 that is, the grade becomes higher, the number of bars 
constituting one unit increases. 

Here, explanation is given by assuming that the unit is 
composed of a plurality of bars, but the construction or the 
number of the bars which constitutes the unit is variable, and 

25 may be composed of not only the bars, but also optional part 
of the music. For example, musically natural training is 
possible, by constituting one unit by one or a plurality of 
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motifs or phrases. In short, it is only necessary to constitute 
the unit so as to include wide range of note information, so 
that as the rank, that is, the skill level becomes higher, the 
player can have training for longer performance at a stretch. 
5 In the case of automatic mode in which the unit is 

automatically designated, the training unit is automatically 
designated according to the predetermined progress. For 
example, the skill level is automatically judged by the PC 1 
based on the performance result, to designate the unit for next 

10 performance, taking into consideration that the player does not 
train repetitively for the unit in which the skill level has 
reached the acceptable standard. On the other hand, in the case 
of manual mode in which the unit is manually designated, the 
player him/herself can selectively designate the unit which 

15 he/she wants to have training. The manual mode may be released 
to return to the automatic mode, or the lesson may be interrupted 
during the automatic mode to optionally designate a unit. A 
data for generating a mode selection display on a screen can 
be included in the performance data so as to be displayed on 

20 the display 14 at a predetermined timing. 

Fig. 3 is a diagram showing one example of the lesson menu 
displayed on the display 14, the diaphragm includes units for 
each rank corresponding to the score. In Fig. 3, one or plural 
unit u which has various size are set for each rank, with respect 

25 to the score for 12 bars displayed on the display 14. When the 
music is longer than the length of one screen, the part on the 
right side where the part without the clef, the key signature 



and the time signature is scrolled from the right to the left 
on the screen, to thereby display up to the last part of the 
music score. Alternatively, instead of scrolling on the screen, 
the part of the music or the whole part may be displayed, by 
5 enlarging or reducing the display size. For example, displayed 
buttons " + " and "-" on the left bottom corner on the screen can 
be operated to scale the screen. 

The skill level in the performance is designated as rank 
Rl, R2, R3, R4 and R5 in order of from the higher rank, and the 

10 unit in which the size, that is, the number of bars included 
therein, is changed is set corresponding to each rank. In the 
lowest rank R5, two bars constitute one unit, and in the next 
higher rank R4, four bars constitute one unit. In this manner, 
as the rank becomes higher, the number of bars constituting one 

15 unit increases. 

The lesson menu is a list, in which the result based on 
the performance result can be seen at a glance, and is also a 
display of the next training unit. When the music performance 
of the designated unit has finished, the performance result is 

20 compared with the prepared model performance data, to judge the 
result. The result is displayed in the lesson menu with a 
character representing the grade, and the unit which has not 
reached the acceptable standard is designated again. 

In the example shown in Fig. 3, it is displayed with a 

25 stripe of spot pattern that the training of unit Ul consisting 
of the first bar and the second bar in rank R5 has finished, 
and unit U2 consisting of the third bar and the fourth bar is 
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displayed with a frame as the next training unit. The result 
is displayed in the unit already played. The result is 
displayed with characters of s, A, B, C and D in order of higher 
grade. in the units ui and U3 having reached the acceptable 
standard, the character of grade "S" and a band indicating 
acceptance are displayed. However, this classification of 
result is an example only, and the number of grades can be 
optionally set, and the acceptable standard can be elevated, 
as the rank becomes higher. When the acceptable standard has 
not been reached, the designation of the unit is not updated, 
and the same unit is designated again. 

In the mode of automatically designating the unit, a unit 
to be played at the earliest timing in the same rank, of the 
units which has not reached the acceptance mark, is designated. 

Since the result of the unit having the same musical tone 
information is managed collectively, as shown in the Fig. 3, 
when the unit Ul has reached the acceptance mark, the unit U3 
consisting of the ninth and the tenth bars having the same music 
tone information or note data array as that of the unit Ul is 
regarded as having trained already, and the result including 
the stripe and the character "S" indicating the evaluation is 
displayed. Therefore, it is avoided to have repetitive 
training of the mastered unit. 

When a unit is designated, the key depression instruction 
described later is displayed on the display 14, based on the 
prepared performance data of the designated unit. The player 
can repetitively train according to the display of the key 
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depression instruction, until the skill level of performance 
is enhanced for each unit, that is, a predetermined acceptance 
judgment is obtained. 

When all units are judged as being acceptable with respect 
5 to the current training rank, the player can proceed to the next 
higher rank, where a unit having a larger size, that is, a unit 
having a large number of bars is designated, and the performance 
data for this unit is played. For example, when the player has 
come up to the standard in rank R5, the player proceeds to rank 

10 R4, where unit U4 is designated, and the performance data is 
played. In rank R4, the player has training for the number of 
bars twice the number in rank R5 at a stretch or without 
interruption. As the rank becomes higher, the degree of 
difficulty, that is, the tempo and the acceptable standard is 

15 increased, thereby training of higher degree corresponding to 
the skill level becomes possible. Judgment of the player's rank 
will be described later. 

On the other hand, in the case of training in the manual 
mode, the unit is designated selectively, by the player's own 

20 will, to play the performance data. Therefore, training can 
be performed repetitively until the trainee is satisfied. 
However, if the player can totally optionally designate the unit 
in the manual mode, the training may not be result ful, and hence 
it is desired that the rank cannot be elevated until the 

25 performance result reaches the allowable level . In other words , 
until all units in the current rank reach the acceptance mark, 
only the unit in the current rank can be designated, and the 



unit in the higher rank cannot be designated. 

Each unit can have the data structure described below. 
That is, unit result information, link information and result 
information can be included as the data of each unit. 
5 The unit result information shows training result for 

each unit, and can include high score, average score, registered 
date of result, and the like. The unit having the same musical 
tone information is managed with the common unit result 
information, with a unique ID enabling identification of the 
10 unit. 

The link information is registered for each unit, and 
having the lead position and the tail end position (both are 
set as time information) of the unit based on the beginning of 
the music, and a link ID which links a plurality of units having 

15 the same musical tone information. The result information has 
array information of the link information for each rank and each 
unit in the entire music. 

The processing by the musical performance self-training 
apparatus will be described, with reference to the flowchart. 

20 Fig. 4 is a main flowchart. In step SI, initialization of the 
PC 1 including clear of a VRAM, which stores image data for 
displaying an image on the display 14, and clear of timer counter 
is conducted. In step S2, a music is selected. For example, 
a list of music is displayed on the display 14, and the player 

25 selects one from the list, by operating the keyboard 12 and the 
mouse 13. After selection of the music, process proceeds to 
step S3, where a unit is designated. Designation of unit is 

25 



for designating which part of the performance data of the 
selected music is a training object, and a lesson menu based 
on the performance data is displayed on the display 14, and the 
training unit is specified thereon. An example of the lesson 
5 menu in which a unit is set is shown in Fig. 3. Designation 
of the unit will be described later with reference to Fig. 5. 

In step S4, the lesson is started. The lesson may be 
started automatically, or start instruction may be given by 
using the keyboard 12 and the mouse 13. In step S5, the key 
10 depression instruction is displayed on the display 14 according 
to the performance data. The display example of the key 
depression instruction will be described later with reference 
to Fig. 6. 

In step S6, musical tones included in the performance data, 
15 that is, accompaniment and metronome sound are played. The 
played sound can be produced using the tone generator 24 and 
the sound system 22 in the keyboard instrument 2. In step S7, 
the performance results of the player, that is, the key 
depression time, the key release time, the velocity and the like 
20 are read in the PC 1 , for evaluation of the performance and for 
advancing a pointer for reading the performance data for the 
key depression instruction. It is desired to perform the 
process of steps S5 to S7 by interrupting handling. When the 
pitch (represented by a note number) designated by the played 
25 performance data and the pitch (represented by a depressed key) 
in the performance result do not coincide with each other, 
reproduction of the performance data is not advanced to the next 



26 



20 



25 



stage, that is, the pointer is not advanced. 

In step S8, it is judged whether the lesson is finished. 
When performance for all of the performance data of the 
designated unit has finished, the judgment in step S8 becomes 
5 affirmative, and process proceeds to step S9. when the lesson 
is not finished, process proceeds to step S5. 

in step S9, result processing or evaluation is performed, 
in the result processing, the performance result read in step 
S7 , that is , the depressed key data is compared with the prepared 
10 performance data, and the result is judged according to the 
degree of coincident. The comparative contents include the 
length of the note (duration of key-on), the velocity, the key 
depression timing and the like, and the number of miss-touch, 
that is, the number of key depression in which different key 
from the designated key by key depression instruction is 
depressed may be compared with the number of the acceptable 
standard, in the result processing, the grade is determined 
based on the comparison results. 

In step S10, as a result of the result processing, it is 
judged whether the performance of the designated unit by the 
player reaches the acceptable standard, if the unit has reached 
the acceptable standard, process proceeds to step S3 , to provide 
a unit to be played next . in the case of nonacceptance, process 
proceeds to step Sll, to judge whether the training is to be 
continued. This judgment can be made based on the existence 
of instruction by the player, using the keyboard 12 and the mouse 
13. if the training is to be continued, process proceeds to 
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step S4. 

Fig. 5 is a flowchart for designating a unit (step S3). 
In step S31, it is judged whether designation process is 
executed by automatic mode or manual mode. An instruction 
5 information that instructs automatic designation or manual 
designation may be included in the performance data to omit 
giving an instruction operation to indicate automatic or manual. 
In the case of automatic mode designation, process proceeds to 
step S3 2, and in the case of manual mode designation, process 

10 proceeds to step S33, to display a lesson menu. In step S33, 
it is desired to display a message to call a player's attention 
for giving instruction, such as "Please designate a unit", on 
the lower part of the screen, together with the display of the 
lesson menu shown in Fig. 3. In the initial menu, the first 

15 unit in the lower most rank is designated. If the player likes 
it better as it is, the player inputs start instruction. When 
the player wants to change the initial menu, he or she designates 
a unit as desired by indicating the frame of unit on the menu 
by using the mouse 13 and the like. In step S5, the performance 

20 data is read according to the link information of the unit, and 
the key depression instruction is displayed corresponding to 
the performance data. 

In the case of automatic designation, process proceeds 
from step S31 to step S32, to judge whether the performance for 

25 all units in the highest rank has come up to the passing mark. 
If the player has not passed the highest rank, process proceeds 
to step S34, to judge whether the performance of all units in 
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the current rank is acceptable based on the unit result 
information. If this judgment is negative , process proceeds 
to step S36, to designate the training unit to be played next. 
For example, as shown in Fig. 3, the display is changed such 
5 that unit U2 is enclosed by a frame, and then process proceeds 
to step S4 (Fig. 4) . If the judgment in step S3 4 is affirmative, 
process proceeds to step S35, to elevate the rank by one stage, 
and process proceeds to step S36. For example, when the rank 
is elevated by one stage from rank R5 to rank R4 , in step S3 6 

10 immediately thereafter, a unit consisting of the first four bars 
in rank R4 is designated. In this manner, when the player is 
passed the highest rank Rl, the judgment in step S32 becomes 
affirmative, to finish the processing. 

Fig. 6 is a diagram showing one example of the key 

15 depression instruction displayed on the display 14 in step S5 
shown in Fig. 4 . In the Fig. 6, the keyboard figure is displayed 
on the upper part and the lower part of the screen. In order 
to make it easy to see to which key on the keyboard the key 
depression instruction corresponds, the keyboard figure K is 

20 displayed on upper and lower area of the screen, but this figure 
(Fig. 6) may be displayed only on the lower area of the screen. 
A plurality of substantially rectangular marks M having a length 
corresponding to a length or a vertical size of the note, and 
a width of the white key in the keyboard figure K, and displayed 

25 between the keyboard figures K and K, is the key depression 
instruction. One mark corresponds to one note. On the display, 
the vertical direction indicates a time axis, and as the key 
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depression instruction mark becomes closer to the lower 
keyboard figure K, it shows a note to be played at the earlier 
timing. The key depression instruction mark M moves downward 
at a predetermined tempo, and it is the timing to play the key 
5 immediately below the mark when the lower end of each key 
depression instruction mark M reaches the upper edge of the 
lower keyboard figure K. On the other hand, it is the key 
release timing of the key when a key depression instruction mark 
corresponding to the key now being depressed is scrolled 

10 downward and disappears from the screen. The movement, that 
is , scrolling of the key depression instruction mark is executed 
so that the key depression instruction mark for the note to be 
played next abuts against the upper edge of the lower keyboard 
figure, when the player plays the key as instructed. The scroll 

15 may start at the time of key release, instead of key depression. 

As the display method of the key depression instruction, 
for example, one described in Japanese Patent Application No. 
2001-352206 according to the application by the present 
applicant may be applied. Display of the key depression 

20 instruction is not limited to the scroll type in which musical 
parts are sequentially scrolled, and the entire music may be 
instructed at the same time, or scrolling or switching of the 
screen may be carried out so that the predetermined number of 
bars is displayed on one screen, to proceed the display forward. 

25 Moreover, when the key depression instruction is scrolled, the 
velocity thereof is optional, and may be according to the tempo 
of the music. 



Fig. 1 is a block diagram showing the main function of 
the PC 1 for designating the unit. In this figure, the result 
information for a performance of the music related to each unit 
is stored in a result storage 3, every time the performance of 
5 each unit finishes. A unit acceptance judge 4 judges whether 
the performance of all units in the current rank reach the 
acceptable standard by comparing the information stored in the 
result storage 3 with the prepared standard performance data. 
When all the units which is performed are accepted, the judgment 

10 result is inputted to a rank-up instruction section 5. The 
rank-up instruction section 5 responds to the input judgment 
result, and outputs a rank-up instruction to a lesson menu 
generator 6. By this rank-up instruction, the lesson menu 
generator 6 generates data for displaying menu including 

15 designation of the first unit in the rank upper than the current 
rank by one stage as a unit to be played next, and the display 
14 displays a lesson menu responding to the data provided by 
the lesson menu generator 6. For example, a unit to be played 
next is enclosed by a thick frame, as the unit U2 shown in Fig. 

20 3. 

The unit is designated on the lesson menu, and the 
performance data representing the score in the designated unit 
is read out from performance data storage 7 to a key depression 
instruction generator 8. The key depression instruction 
25 generator 8 generates key depression instruction information 
based on the performance data and outputs the information to 
the display 14, to display an image shown in Fig. 6. 
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When the unit acceptance judge 4 judges that only 
performance of a part of the units in the current rank is 
acceptable, the judgment result is input to a unit update 
section 9. The unit update section 9 responds to the input 
judgment result, and outputs a unit update instruction to the 
lesson menu display section 6. The lesson menu generator 6 
outputs data having a designation where the unit subsequent to 
the currently designated unit is the unit to be played next, 
according to the unit update instruction . Since the unit having 
the same content is controlled collectively, at the time of 
updating the unit, the unit having the same musical tone 
information as that of the unit that has been accepted is not 
designated, and a unit to be played next is designated. 

The display format of the lesson menu is not limited to 
the image corresponding to the score, as shown in Fig. 3. Fig. 
7 is a diagram showing an example of the lesson menu, in a mode 
in which the score is represented by a keyboard figure and a 
mark corresponding to the keyboard figure (referred to as a 
piano roll view mode). The keyboard figure K is arranged in 
the lower part of the screen, and above this keyboard figure 
K, a time axis is elongated vertically, and a rectangular mark 
M having a length (size along to the time axis) corresponding 
to the length of the note is displayed. On the right side of 
this mark, a unit U for each rank is displayed. 

in Fig. 7, there is shown an example in which the unit 
including the first to the fourth bars in rank R5 is performed 
acceptably, and the unit including the fifth to the eighth bars 
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ia performed unacceptably (grade D), and hence the unit 
consisting of the fifth and sixth bars is designated again. 

in the above-described embodiment, the unit is designated 
such that when the performance of one unit is executed 
5 acceptably, the next bar is selected in the same rank to be played 
However, the designating method is not limited to this, and the 
unit may be set so that the player can have continuous training 
for many bars at a stage as early as possible. 

For example, in Fig. 3, when unit ul and the unit adjacent 
10 to unit Ul are accepted, the next unit in the same ranx is not 
designated next, but the rank is elevated by one and the first 
unit in the upper rank R4 may be designated, since the first 
unit in rank R 4 consists of the first two units in rank R5, 
relatively long performance can be experienced in rank R4 where 
15 training for these two units is performed in succession. 

When the performance of the first unit in rank R4 is 
accepted, returning to rank R5 again, the third unit is selected 
When the performance of this unit is accepted, the fourth unit 
m rank R5 is selected, when the performance of the fourth unit 
20 in rank R5 is accepted, the second unit in rank R4 is selected. 
When the performance of the second unit is accepted, the first 
unit in rank R3 is selected. m this manner, when the 
performance of the unit in the lower rank included in one unit 
of the upper rank is accepted, training can be performed by 
5 Proceeding to the upper rank in order to play longer bars of 



music. 



The key depression instruction is set such that unless 
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the key as instructed is not piayed. scroll is not carried out, 
in order to improve the skill level. However, i„ the highest 
rank HI, the skill level of the piayer should be improved. 
Therefore, even i, the key is not played as the key depression 
5 instruction indicates, the key depression instruction 
information, that is, the mark may be scrolled, it is for 
enabling through training of the entire music. 

The unit may not be fixed. The borderline between units 
set in the performance data is set to a sise, which is considered 
10 to be desirable for many users. However, this setting may be 
difficult to play for some users. For example, when a player 
wants to practice for a long part at a stretch, if the unit is 
fixed, the player must have playing of a higher rank having a 
larger unit size, m such a case, if the player can change the 
15 borderline to change the size o, the unit, only the length to 
be played at a stretch can be changed without changing the rank, 
which is preferable. 

Moreover, when a unit is set in advance in the performance 
data, for example, it is easier to divide the unit by two bare 
0 mechanically. However, for example, in the case of a music 
starting with up beat (Auftakt, , in which a break in performance 
does not exist in the bar line, a unit divided by the bar is 
inconvenient, m such a case, it is guite convenient if the 
borderline between units can be shifted. 

Pig. 8 is a diagram showing a setting example of units 
which is not appropriate. This figure shows a music starting 
wrth an Auftakt or up beat (name of the son,, Hotaru no Hikari 
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in Japanese), and the units are set such that two bars simply 
constitute one minimum unit. This song is well known, and it 
can be considered that a beginner may practice while humming 
the lyrics. With the division of units shown in this figure, 
however, it does not correspond to the syllabic for Japanese 
pronunciation, such as "ho-ta-runo-" and "hi-kaa-ri-" , and 
hence it is difficult to play. For a reference, lyrics 
(alphabetical notation of Japanese) is shown corresponding to 
the note in the figure. 

Therefore , the borderline between units is reset, so that 
practice can be done by a unit division corresponding to the 
syllabic for Japanese pronunciation. In this manner, since the 
range to be played at a stretch corresponds to a unit, it becomes 
easy to practice especially for a beginner. 

Fig. 9 is a diagram showing an example in which the 
borderline between units is reset. As described above, the 
division of units is changed, and the lesson menu is created 
so that playing according to the lyrics of "Hotaru no Hikari" 
is possible. 

This example shows a simple up beat music, for easy 
understanding. Actually, however, there is various 
complicated music, and it is difficult to set the units fully 
appropriately automatically by a computer. Therefore, it is 
meaningful to have a function of resetting the division of 
units . 

Fig. 10 is a diagram showing a lesson menu of another music, 
in which it is desired to reset the units. In this music, a 
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player must proceed to rank Rl, in order to have continuous 
practice for the eighth and the ninth bars. Therefore, for 
example, in rank R4, a unit including the eighth and the ninth 
bars is set, so that the player can have practice for the eighth 
5 and the ninth bars continuously. In this manner, one playing 
range can be reset in the lower rank according to preference, 
thereby enabling effective practice. 

In order to change the size of the unit, for example, a 
player specifies a borderline (vertical line for providing 

10 unit) between units which the player wants to shift by the mouse 
13, and draws the borderline to an optional position. Other 
than this method, a borderline between units may be added newly, 
or an existing borderline may be deleted. In this case, the 
borderline is rounded off to a vertical line (a line roughly 

15 dividing for each one note) shown on the score. As the method 
for editing the line in the figure, a known drawing method with 
a computer can be employed. 

When the unit is reset, the area for managing the result 
is also changed. For example, in an example shown in Fig. 10, 

20 when the eighth and ninth bars are designated as one unit, result 
management is performed for this unit. 

In the above embodiment, an example in which the key 
depression instruction and lesson menu are displayed according 
to the performance data is shown . However, the performance data 

25 may be used as data for auto-playing. In this case, a switch 
for starting model reproduction is included in the lesson menu, 
and by instructing this switch by the mouse 13, model 
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performance is performed according to the performance data 
included in the training part, that is, the designated unit. 
The model performance may automatically finish when the 
reproduction, that is, performance of the performance data in 
5 the unit has finished, or a switch for stopping the model 
reproduction may be provided in the lesson menu. 

Moreover, the performance result can be listened and 
confirmed. In this case, a recording/reproduction switch is 
provided in the lesson menu, and the performance result is 
10 recorded in a memory in the PC body 11, and the recording result 
can be played. Model reproduction and reproduction of the 
performance result are performed, by using a sound unit 
connected to the keyboard instrument 21. 

Fig. 11 is a flowchart showing the second embodiment of 
15 this invention in which the designation of the unit can be 
changed while playing. 

In the following description concerning to Fig. 11, same 
or similar operation is omitted to describe for avoiding 
repetition. 

20 In step S103, it can be judged whether the training mode 

is the automatic mode or the manual mode based on the instruction 
information. An image for prompting an instruction of 
designation mode may be displayed on the display 14, so that 
a player can select the mode. In this case, the player can 

25 select the mode from the displayed selection image, using the 
keyboard 12 and the mouse 13. 

When the automatic mode is instructed, process proceeds 
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to step S104 f to automatically set the lesson content. In other 
words, a unit for which the performance data is played is 
automatically set, according to the predetermined procedure. 
The automatic designation of unit has previously described in 
5 Fig. 5. 

In the case of manual mode, control proceeds to step S105, 
to display the lesson menu shown in Fig. 3. 

In step S106, it is judged whether a start instruction 
has been input. When the start instruction is given, process 

10 proceeds to step S107. An icon for start instruction may be 
included in the lesson menu. In step SI 07, it is judged whether 
the training mode is the automatic mode or the manual mode. When 
process proceeds to step S105 after interrupting the lesson to 
display the lesson menu, as described later, the mode setting 

15 is confirmed again here. If the unit designation mode is the 
automatic mode, process proceeds to step S104, or if the unit 
designation mode is the manual mode, process proceeds to step 
S108. In step S108, the designated unit is displayed in the 
lesson menu, then content or performance data of the unit is 

20 read out to start the lesson. 

In step S109, a key depression instruction is displayed 
on the display 14 according to the performance data. An example 
of the key depression instruction has described previously in 
Figs. 6 and 7. In step S110, notes included in the performance 

25 data, that is, accompaniment and metronome sound are played. 
In step Sill, the performance results of the player are read 
in the PC 1 for judging the result. Steps S109 to Sill are 



executed similar to the step S5 to S7 of Fig. 4. 

In step SI 12, it is judged whether the lesson is 
interrupted to display the lesson menu. In other words, even 
in the automatic mode, the lesson menu is displayed during 
5 training, by accepting the instruction to display the lesson 
menu, so that the player can designate a unit. If there is an 
instruction to display the lesson menu, in stepS113, the lesson 
is interrupted, and the display of the key depression 
instruction information is stopped, to proceed to step S105. 

10 When an instruction to display the lesson menu is not input 

during a predetermined stand-by time, process proceeds to step 
SI 14, to judge whether the lesson has been finished. When all 
of the performance data of the designated unit is played, the 
judgment in this step SI 14 becomes affirmative. In step S115, 

15 the result processing is carried out. In the result processing, 
the performance result read in step Sill, that is, the actual 
performance data, that is, depressed key data is compared with 
the performance data, and the result is decided according to 
the difference between them. The comparison is executed in 

20 relative to the length, the velocity and the tempo, and the 
number of miss-touch. The number of playing a key different 
from the designated key may be compared with the number in the 
acceptable standard. In the result processing, the rank is 
determined according to the comparison result. 

25 As a result of the result processing, it is judged whether 

the performance of the unit has reached the acceptable standard. 
In step S116, it is judged whether to continue the lesson, based 
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on the presence of instruction of the player input by the 
keyboard 12 and the like. If it is judged to continue the lesson, 
process proceeds to step S103. If there is no instruction to 
continue, this processing finishes. If there is the 
5 instruction to continue, and process returns to step S103, and 
when the performance has reached the acceptable standard, a unit 
to be played next is set, and when the performance has not reached 
the acceptable standard, the same unit is designated again. 
While the lesson menu is displayed, the automatic mode 

10 or the manual mode can be instructed on the lesson menu. For 
example, in the automatic mode, when the player wants to display 
the lesson menu, in step S105, the player can select the mode, 
as well as designating the unit. When the manual mode is 
instructed, process proceeds from step S107 to step S108. When 

15 the unit is selected and the automatic mode is maintained as 
it is, process shifts from step S107 to step S104. When the 
unit is designated and process proceeds to step S104, the lesson 
in the automatic mode is started from the designated unit. When 
the unit designating mode is changed over from the manual mode 

20 to the automatic mode, the lesson is started from the current 
unit in the automatic mode. 

When the lesson menu is displayed to designate a unit, 
while the key depression instruction is given in the automatic 
mode, it is possible not only to jump to a unit ahead, but also 

25 to designate a unit in which the playing has been performed and 
reached the acceptable standard, to improve the performance 
skill level until the player is satisfied. In this case, when 



the performance of the designated unit has finished, practice 
proceeds to a unit to be played at an earlier timing, of the 
units whose performance has not been conducted. The result is 
updated to the newest one. 

Fig. 12 is a block diagram showing the main function of 
the musical performance self-training apparatus according to 
the second embodiment of the present invention. In this figure, 
a lesson menu generator 30 generates a data for displaying the 
lesson menu as shown in Fig. 3. A performance data storage 31 
stores performance data. The lesson menu generator 3 0 
generates data for displaying the menu, by using the performance 
data and the key depression information detected by the 
detection circuit 23, and the menu is displayed on the display 
14 according to the data supplied with lesson menu generator 
30. The lesson menu includes a unit to be played next and the 
result of the played unit (grade and acceptance judgment), and 
various instruction switches (icon) are also displayed, such 
as a unit designation button LI , a mode switch L2 , a start switch 
L3 and the like. For example, by indicating the mode switch 
L2 by the keyboard 12 or the mouse 13, the current unit 
designation mode is inverted. When the mode is the automatic 
mode, it is changed over to the manual mode, and when the mode 
is the manual mode, it is changed over to the automatic mode. 
The unit designation button LI is displayed in the manual mode, 
and by indicating this button LI by the keyboard 12 or the mouse 
13, and instructing to display a desired unit, the unit to be 
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played next is designated. The start switch L3 is provided for 
instructing start of the lesson after the unit has been 
designated. 

When the start switch L3 is pressed, a progress control 
33 reads out the performance data of the designated unit from 
the performance data storage 31, according to the instruction 
input with the unit designation button Ll and the designation 
of the unit, and inputs the performance data to a key depression 
instruction section 34. The key depression instruction 
section 34 displays the key depression instruction on the 
display 14, based on the input performance data. 

When the key depression instruction is to be displayed, 
the lesson menu is deleted from the display 14, and when the 
lesson menu is to be displayed, the key depression instruction 
is deleted from the display 14. The lesson menu is displayed 
every time the performance of the designated unit is finished 
and evaluated, but a lesson interruption switch L4 is provided 
so as to correspond to a case where the player wants to display 
the lesson menu halfway through the performance. This lesson 
interruption switch (icon) L4 can be superimposed on the key 
depression instruction display. When the lesson interruption 
switch L4 is pressed, the key depression instruction displayed 
on the display 14 is replaced with the lesson menu. 

The key depression instruction can be displayed various 
pattern or mode. In an embodiment described below contains a 
means for displaying the key depression instruction for various 
modes to be selected by the player's choice. Fig. 13 is a 
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flowchart showing the third embodiment of this invention. 

In the following description related to Fig, 13, same or 
similar operation is omitted to describe for avoid repetition. 
The performance data of the selected music at step S202 
5 includes data for designating the key depression instruction 
display mode, other than the note information. The display mode 
is designated for each music. 

When the music has been selected, process proceeds to step 
S203 to display a lesson menu on the display 14. The lesson 
10 menu is set such that one or a plurality of parts obtained by 
dividing performance data into a plurality of parts can be 
designated as one unit as described before. The designated unit 
can be extended stepwise corresponding to the progress of 
practice, that is, the parts included in the unit are increased, 
15 so that finally the entire music can be played at a stretch. 
The evaluation of performance can be displayed on the lesson 
menu for each unit. An example of the lesson menu has been 
described with reference to Fig. 4. 

A display mode changeover icon switch is displayed 
20 together with the lesson menu, and the current display mode can 
be replaced with another display mode by operating this switch. 
In step S204, it is judged whether the changeover operation has 
been performed by the display mode changeover switch. If there 
is the changeover operation, process proceeds to step S205, to 
25 change over the key depression instruction display mode. The 
display mode can be changed over every time the changeover 
switch is operated periodically, for example, in order of piano 
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roll view mode/ score view mode and superimpose mode. 

In step S206, it is judged whether a lesson start 
instruction is given or not. The lesson start instruction may 
be given by indicating the icon provided on the image displayed 
5 of the display 14 , or may be input by the keyboard 12. If the 
lesson start instruction is given, process proceeds to step S207 . 
It is possible to change the display mode r since process 
proceeds to step S203 until the lesson start instruction is 
input. If the display mode is not changed over, step S205 is 

10 skipped, and the current display mode is maintained. 

In step S207, lesson start processing is performed. In 
step S208, the key depression instruction is displayed on the 
display 14 in the above-described display mode according to the 
performance data. In step S209, the musical sound included in 

15 the performance data, that is, accompaniment and metronome 
sound are played. The played sound can be produced using the 
tone generator 24 and the sound system 22 in the keyboard 
instrument 2. In step S210, the performance results of the 
player are input to the PC 1, for evaluation of the performance 

20 and for updating a pointer for reading the performance data for 
the key depression instruction. It is desired to perform steps 
S208 to S210 by timer interrupting processing. 

In step S211 and step S212, processing relating to the 
changeover of the display mode by the display mode changeover 

25 switch is carried out, as in step S204 and step S205. 

In step S213, it is judged whether the predetermined 
lesson has been finished, that is, reproduction of the 



performance data of the selected music has been finished. When 
all performance has finished according to the predetermined 
performance data, the judgment in step S213 becomes affirmative, 
and process proceeds to step S214. 

In step S214, the performance result is evaluated. The 
performance is evaluated according to the incidence degree of 
performance result with the musical tone information in the 
performance data. The points to be evaluated include the pitch, 
the length and velocity, and key depression timing. 

In step S215, the evaluation result is displayed on the 
display 14 . The result display example will be described later 
with reference to Fig. 16 and Fig. 17. During this result 
displaying, the display mode can be changed. In other words, 
in step S216 and step S217, processing relating to the 
changeover of the display mode by the display mode changeover 
switch is carried out, as in step S204 and step S205. 

In step S218, it is judged whether the lesson menu is to 
be displayed. The lesson menu display switch or icon for giving 
instruction to display the lesson menu is displayed on the 
display 14 together with the result. When a training unit is 
designated, or the result of each unit is confirmed, the lesson 
menu display switch is operated. When the lesson menu display 
switch is operated, the judgment in step S218 becomes 
affirmative, and process proceeds to step S203 . When the lesson 
menu display switch is not operated, process proceeds to step 
S219 to judge whether to finish the lesson. Instruction to 
finish the lesson can be displayed on the display 14 together 



45 



with the result. If the instruction to finish the lesson is 
given, the processing in this flowchart finishes. 

In this manner, the designation of the key depression 
instruction display mode included in the performance data can 
5 be changed, at any time of starting the lesson, during the lesson, 
and displaying the result. 

When the key depression instruction display mode is 
changed, the key depression instruction display mode set in the 
performance data is updated by the changed key depression 

10 instruction display mode. Therefore, the key depression 
instruction when the music selected this time is again selected 
at the next time is displayed according to the key depression 
instruction display mode updated herein. 

It is preset in the performance data which one of the two 

15 display modes of the lesson menu is to be used, like the key 
depression instruction display mode. However, the superimpose 
mode is not employed for the lesson menu. 

Fig. 14 is a diagram showing an example in which the key 
depression instruction is displayed in the score view mode . The 

20 keyboard figure K is displayed on the lower part of the displayed 
picture. The keyboard figure K may be positioned on the upper 
and lower parts of the displayed image, in order to easily look 
at which key on the keyboard corresponds to the key depression 
instruction. The score is shown above the keyboard figure K, 

25 and the key depression timing is indicated by line L. The score 
is scrolled from the right to the left, and the note overlapping 
on the line L is the note to be played at present. The key 



corresponding to this note is indicated by mark Pi . 

The score may be scrolled for one note, when a key is 
depressed as instructed, or the line L is shifted to the right 
until performance of one grace note finishes, and the screen 
5 may be scrolled by one bar, when the note included in one bar 
has been played at a pitch as instructed • The line L is also 
shifted so as to match with the scrolling. 

Fig. 15 is a diagram showing an example in which the key 
depression instruction is displayed in the piano roll view mode. 

10 The key depression instruction mark M is displayed 
corresponding to the key to be played according to the key 
depression instruction mark, above the keyboard figure K 
displayed on the lower part of the displayed image. Line BL 
is a bar line. In this example, the marks M included in two 

15 bars are displayed at the same time as one displayed image, but 
the size of the one image to be displayed at the same time can 
be preset. One key depression instruction mark M corresponds 
to one note, and each mark M has a length corresponding to the 
length of the note and a width of each key in the keyboard figure 

20 K. On the displayed image, the time axis is in the vertical 
direction, and the key corresponding to a key depression 
instruction mark closer to the keyboard figure K is a key to 
be played at the earliest timing. The mark M is scrolled 
downward. The key which is to be played now is indicated by 

25 mark P2 . 

Scrolling of the key depression instruction mark M is 
executed when the player has played the key as instructed. 



47 



Scrolling may be performed at a key release timing. For the 
display of the key depression instruction, one disclosed in 
Japanese patent Application No. 2001-352206 according to the 
application by the present applicant can be applied. Display 
5 of the key depression instruction is not limited to the 
scrolling method, but the entire music may be indicated at the 
same time, or one displayed picture may be replaced with another 
one. When the key depression instruction is scrolled, the 
velocity thereof is optional, and may be according to the tempo 

10 of the music. 

Fig. 16 is a diagram showing an example in which the 
performance result is displayed in the score view mode, wherein 
the same reference symbol as that in Fig. 14 denotes the same 
or equal part. A check CH is written on the note having bad 

15 evaluation. Annotation corresponding to the check CH, that is , 
the key depression having bad evaluation is described above the 
keyboard figure K. In the case of the picture shown in Fig. 
16, it can be understood that a key with the check CH has been 
miss-touched by the annotation. 

20 Moreover, for easy understanding of the performance 

result of the player, evaluation for a plurality of elements 
is shown by the graph G. The judging elements for evaluation 
include miss-touch, rhythmic sense, tempo, key-on timing, 
key-off timing and stress (strong or weak) expression. 

25 Fig. 17 is a diagram showing the result in the piano roll 

view mode in which the evaluation graph is superimposed pn the 
key depression instruction, wherein the same reference symbol 
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as that in Fig . ls denotes the same or ^ ^ 
same as that in Fig. 16 is shown. 

Fig. 18 is a diagram showing a key depression instruction 
display example in which the dlsplay in ^ p . ^ ^ ^ ^ 

5 is superimposed on the display in the score view mode. The 
length of the note displayed in the score view mode can be 
visually understood in the piano roll view mode. By comparing 
the displays in the both modes, even a beginner who cannot read 
the score can understand the meaning of the sign or note on the 
10 score, and can understand how to play according to the sign and 



note. 



Fig. 19 is a diagram showing another example of the 
superimpose mode. In this display mode, the keyboard figure 
K is arranged in the upper part, and the staff St is described 

15 corresponding to the keyboard figure K. The notes and the key 
depression instruction mark H are described on the staff ST. 
in this manner, since the staff ST is displayed longitudinally 
so as to correspond to the keyboard, the relation between the 
key position and the staff can be understood, and hence the 

0 relation between the note to be played and the key can be well 
understood. 

The keyboard figure K is displayed long sideways, linked 
with the actual playing posture, but when priority is given to 
the arrangement of the known staff, the keyboard figure K may 
be arranged longitudinally. Moreover, also in the superimpose 
mode, the performance evaluation may be displayed together. 
Fig. 21 is a block diagram showing the main function for 
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changing over the key depression instruction display mode. A 
performance data readout instruction section 40 outputs a 
readout command to a performance data storage 41 responding to 
a lesson start instruction or key depression at a correct pitch. 
5 In response to the readout command , the performance data storage 
41 outputs event data and timing data. The performance data 
is read out in order of from the top. The performance data is 
input by any one of a first key depression instruction display 
section (score view mode display section) 42, a second key 

10 depression instruction display section (piano roll view mode 
display section) 43, and a third key depression instruction 
display section (superimpose mode display section ) 44, 
corresponding to the condition of the changeover section 45, 
which is changed over according to the instruction input by the 

15 player. Moreover, note data, that is, data indicating the pitch, 
of the event data, is also input to a key depression detection 
section 46. 

The key depression detection section 46 compares the 
depressed key data detected by the detection circuit 23 with 

20 the note data, and outputs a detection signal when it detects 
that the correct key has been played. A first gate (OR) 47 opens 
to inputs the detection signal to a second gate (AND) 48, when 
the lesson start instruction is input, or the detection signal 
is input, and in response thereto, a readout command is input 

25 to the performance data storage 41. 

Every time the key depression detection section 46 
detects that the correct key has been played as instructed, the 



next performance data is read out to the first to the third key 
depression instruction display sections 42 to 44, and the key 
depression instruction is displayed in the predetermined 
respective modes, based on the input performance data. 
5 In the above embodiment, three display modes are changed 

over and used, but the present invention is not limited thereto, 
and it is only necessary that at least either one of the score 
view mode and the piano roll view mode can be selected and used. 
Fig. 22 is a flowchart showing the fourth embodiment of 
10 this invention. In order to avid repetition of description, main 
portion of the embodiment is described below, referring to 
Fig. 22. 

The performance result fetched into the PC body 1, then 
is written in a memory such as a RAM. The pitch in the 

15 performance result is compared with the performance data one 
by one,, and when the instructed pitch in the performance data 
coincides with the pitch in the performance result, 
reproduction of the performance data proceeds forward, thereby 
to advance the performance instruction. However, when the both 

20 pitches do not coincides with each other, reproduction of the 
performance data cannot proceed forward. The performance 
result in which the pitch does not coincides with the pitch in 
the performance data is overwritten with the new data, that is 
deleted, at the point in time when the performance result in 

25 which the pitch coincides with the pitch in the performance data 
is fetched. Steps S5 to S7 are carried out by timer interrupting 
processing. 
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In step S308 , it is judged whether the lesson has finished. 
When performance for all of the performance data of the 
designated unit has finished , that is, when the data ends f the 
judgment in step S308 becomes affirmative, and process proceeds 
5 to step S309. When the lesson has not finished, process 
proceeds to step S3 05. 

In step S309, result judgment is performed. In the result 
judgment, the performance result read in step S307, that is, 
the actual key depression result by the player is compared with 

10 the performance data, and the result is judged according to the 
degree of agreement. The comparative contents include the 
key-on time, the key-off time and the velocity. Since only the 
performance result whose pitch coincides with the instructed 
pitch is stored, the pitch is not included in the result judgment. 

15 The agreement with the pitch is a premise of the results to be 
judged here. However, when a note is played at a pitch different 
from the instructed pitch, it is counted as a miss-touch, 
compared with the number in the acceptable standard and 
reflected in the result judgement. By these comparison results , 

20 the tempo evaluation, on-timing evaluation, off -timing 
evaluation, rhythm evaluation, volume evaluation, and 
miss-touch evaluation are individually performed, and the rank 
S to D is determined based on the comprehensive judgment. These 
evaluation items are one example only, and evaluation may be 

25 performed more finely, or may be performed with respect to fewer 
items . 

In step S310, the result of performance evaluation is 
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displayed. The rank may be displayed on the lesson menu, and 
the comprehensive evaluation obtained by integrating the 
individual evaluations may be superimposed on the key 
depression instruction display by using a graph, separately 
5 from the rank display. In step S311, as a result of the result 
judgment, it is judged whether the unit has come up to the 
acceptable standard. If the unit has come up to the acceptable 
standard, process proceeds to step S303, to designate a unit 
to be played next. If the performance of the unit has been 

10 passed in the judgement, process proceeds to step S312 to judge 
whether to continue the lesson. If the lesson is to be continued, 
process proceeds to step S303. On the other hand, when the 
performance of the unit has not been passed in the judgement, 
process proceeds to step S3 13 to judge whether to continue the 

15 lesson. The judgment of continuation of the lesson can be 
performed based on the existence of the lesson continuation 
instruction from the player by using the keyboard 12 or the like. 

Fig. 23 is a diagram showing one example of the performance 
data. The performance data is compared with the performance 

20 result in the progress judgment of the key depression 
instruction for each key depression, and in the result judgment 
at the time of finishing the performance of the designated part. 
In this figure, the note number (provided in MIDI standard), 
model key-on time, model key-off time and model velocity are 

25 set with respect to the instructed keys Al to AN arranged in 
the order of performance. 

Fig. 24 is a diagram showing one example of the performance 
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results stored in the memory or storage- The input key-on time, 
input key-off time and input velocity are written in the memory, 
relating to the depressed keys Bl to BN with respect to the 
instructed keys Al to AN. The performance result is written 
5 for each tone in real time* When music is played with a tone 
where a pitch is different from the pitch of the model note number, 
it is counted as a miss-touch, and the number of miss-touches 
(input errors) is written for each playing. 

Fig. 25 is a diagram showing one example of sub-evaluation, 

10 which becomes the grounds for the comprehensive evaluation. In 
this figure, the tempo ratio stands for a value of ratio between 
the time from the model key-on time of the instructed key Al 
to the model key-off time (which may be the model key-on time) 
of the instructed key AN, and the time from the input key-on 

15 time of the depressed key Bl to the input key-off time (which 
may be the input key-on time) of the depressed key BN. In other 
words, the tempo ratio is an index indicating how much the actual 
performance time is longer than the model performance time, or 
how much the actual performance time is shorter than the model 

20 performance time. 

An adjusted key-on time and adjusted key-off time stand 
for a value obtained by dividing the input key-on time and the 
input key-off time by the tempo ratio, respectively. In other 
words, these are values obtained by expanding or contracting 

25 the input key-on time and the input key-off time according to 
the tempo ratio, so that the key-on time by the performance data 
and the actual key-on time can be compared with each other. 



Key-on timing evaluation is the evaluation corresponding 
to a difference between the model key-on time and the input 
key-on time, and key-off timing evaluation is the evaluation 
corresponding to a difference between the model key-off time 
5 and the input key-off time. Evaluation in the range of from 
1 to 5 is given according to the difference. 

Step time evaluation is the evaluation judged by 
comparing the step time with the model step time. By this step 
evaluation, it can be evaluated whether the pause between two 
10 tones has been taken well. 

Gate time evaluation indicates a value judged by 
comparing the key-on duration, that is, the time from the input 
key-on time to the input key-off time with the time from the 
model gate time. 

15 Adjusted key-on evaluation, adjusted key-off evaluation, 

adjusted step time evaluation and adjusted gate time evaluation 
are expanded and contracted values, obtained by respectively 
performing the similar evaluation to the input key-on timing 
evaluation, input key-off timing evaluation, step time 

20 evaluation and gate time evaluation. 

Fig. 26 is a diagram showing one example of the 
comprehensive evaluation based on the sub-evaluation. The 
comprehensive evaluation is obtained by comprehensively 
judging the tempo, key-on timing, key-off timing, rhythm, sound 

25 volume and miss-touch for tones corresponds to all depressed 
keys. For the tempo evaluation, higher evaluation is given as 
the tempo ratio is closer to "1". The key-on timing evaluation 
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is a value obtained by determining the highest value of the 
key-on timing evaluation, step time evaluation, adjusted key-on 
evaluation and adjusted step time evaluation, for each tone 
number relating to the tone corresponds to the depressed keys 
5 Bl to BN, and averaging the highest values thereof. 

Similarly, the key-off timing evaluation is a value 
obtained by determining the highest value of the key-off 
evaluation, gate time evaluation, adjusted key-off evaluation 
and adjusted gate time evaluation, for each tone relating to 

10 the depressed keys numbers Bl to BN, and averaging the highest 
values thereof. 

Rhythm evaluation becomes higher, as there are more 
expanded and contracted key-on evaluations having come up to 
the acceptable standard. Moreover, volume evaluation becomes 

15 higher, as the difference between the model velocity and the 
input velocity becomes smaller. Miss-touch evaluation becomes 
higher, as the number of miss-touches decreases. 

The above evaluation is divided into five grades, but the 
division of evaluation is not limited thereto, and for example, 

20 evaluation may be made out of a hundred, or three-grade 
evaluation may be made. 

Fig. 27 is a diagram showing one example of key depression 
instruction display including a graphic display of the 
comprehensive evaluation. In this figure, a keyboard figure 

25 is displayed on the lower part of a displayed image. In order 
to make it easy to look at which key on the keyboard the key 
depression instruction corresponds, the keyboard figure K may 
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be displayed also on the upper part of the displayed image. 
Rectangular key depression instruction marks M having a long 
side representing the length of the tone and a narrow side having 
the same length as the width of the key in the keyboard figure 
K are displayed above the keyboard figure K. One key depression 
instruction mark M corresponds to one tone. Here, key 
depression instruction marks for two bars are displayed. On 
the displayed image, the vertical direction corresponds to a 
time axis, and the key depression instruction mark M closest 
to the lower keyboard figure K indicates the key to be depressed 
at the earliest timing. The key depression instruction mark 
M moves downward sequentially, and a key corresponding to the 
key depression instruction mark M whose lower end has reached 
the lower keyboard figure K is the key to be depressed next. 
Scrolling of the key depression instruction mark is executed 
when the player depress the key corresponding to a right pitch. 
However, the scroll timing is not limited thereto. For example, 
at the time of key-off, the key depression instruction mark M 
may be scrolled by one. 

The comprehensive evaluation G is displayed together on 
the screen where the key depression instruction mark M is 
displayed. It is desired that the comprehensive evaluation be 
such that the ability of the player can be intuitively 
understood by the shape profile of a polygonal graph and a 
deviation of the central point, as in this figure, but the type 
of graph is not limited thereto. For example, by a bar chart, 
it can be easily recognized which evaluation element is the 
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strong point or the weak point of the player. 

Graphic display is not limited to use for representation 
of the comprehensive evaluation. For example, an optional item 
in the comprehensive evaluation is selected , and the evaluation 
for each key depression can be displayed for the selected item 
in a time-series line chart. Thereby, a problematic point in 
the process of progress of performance can be recognized. The 
displayed item may be selected by instructing it by the mouse 
13 or the like on the displayed graph for the comprehensive 
evaluation. The graph may be displayed in a space where the 
key depression instruction mark M is not described, or may be 
displayed as another image separate from the key depression 
instruction image displayed. 

Fig. 28 is a block diagram showing the main function of 
the musical performance self -training apparatus related to the 
fourth embodiment. In this figure, performance data storage 
50 stores performance data of music for performing the key 
depression instruction. A key depression instruction 
generator 51 generates information for displaying the key 
depression instruction according to the performance data. A 
key depression detection section 52 detects the depressed key 
number, key-on time, key-off time and the velocity, based on 
a detection signal of a sensor or a switch provided in the 
keyboard 21. When the depressed key number coincides with the 
note number in the performance data, the key depression 
instruction generator 51 reads out the performance data 
relating to the next tone from the storage 50, and updates the 
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information for displaying the key depression instruction. 
The information for displaying the key depression instruction 
is output to the display 14 and the key depression instruction 
is displayed. 

5 A depressed key storage 53 fetches the output data from 

the key depression detection section 52 and stores the data. 
A result judgment section 54 evaluates the performance by 
comparing the depressed key information stored in the depressed 
key storage 53 with the performance data. A result chart 
10 generator 55 creates a chart such as a graph based on the 
performance evaluation. The created chart is displayed on the 
display 14 together with the key depression instruction or 
separately. 

The result judgment section 54 does not judge the result 
15 only by the fact that a note has not been played at the instructed 
pitch (it is the precondition of result judgment that notes are 
played at the instructed pitch) , but evaluates the performance 
by comparing the information mainly relating to the performance 
timing, such as the overall performance time, key-on time, 
20 key-off time, gate time and step time, with the model data and 
the input data. Therefore, the player's ability can be 
evaluated in detail, and the grade can be determined based on 
the evaluation. 

As described above, according to this embodiment, the 
25 player's ability can be specifically judged. Based on the 
judgment result, the evaluation standard can be changed. For 
example, ranks Rl to R5 have respectively different evaluation 
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standard. If the evaluation standards of those ranks are linked 
with the evaluation result by the above-described evaluation 
method, the rank becomes adequate to any player. As a result, 
if the player has training sequentially from the lower rank, 
5 he/she can enhance the ability reasonably. 

Fig. 29 is a flowchart showing the input processing of 
the performance result for an another example of result 
evaluation. In step S460, it is judged whether the key 
depression is performed. If key depression is detected, process 

10 proceeds to step S461, to judge whether a played note storage 
area (buffer) for storing the played note or the depressed key 
information is cleared. If the buffer is vacant, process 
proceeds to step S462. On the other hand, if at least one 
depressed key data is stored in the buffer, process proceeds 

15 to step S466. 

In step S462, the pitch of the tone related to the key 
depression is examined. In other words it is judged whether the 
pitch of the performance data for generating the key depression 
instruction indicated by a key depression instruction pointer 

20 is identical with the pitch of the tone corresponds to the 
depressed key. In this embodiment, the pitch of each data or 
tone can be represented by a key number and a note number 
respectively, for simplifying the comparison process . If these 
are identical, process proceeds to step S463, to evaluate the 

25 performance by comparing the depressed key and the key 
depression instruction in the performance data indicated by the 
key depression instruction pointer. In step S4 64, the 
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evaluation is stored in an evaluation storage area described 
later. The evaluation of performance can be judged, for example, 
depending on how much the key-on timing is deviated between the 
performance data and the depressed key. The key-on timing is 
5 represented, for example, by a step time, that is, a duration 
time from the bar line, and measured by a counter. In step S465, 
the key depression instruction pointer is advanced forward. By 
updating the key depression instruction pointer, a further 
performance data is read out to generate a new key depression 

10 instruction mark, and the current key depression instruction 
mark is scrolled out. 

When the pitch of the note (note number) indicated by the 
key depression instruction pointer and the pitch of the 
depressed key tone (key number) are not identical, process 

15 proceeds from step S462 to step S466. In step S466, the 
depressed key data, for example, the pitch (key number) and the 
key-on timing are stored in the buffer. In step S467, it is 
judged whether a predetermined number of data has been stored 
in the buffer. It is for judging whether there is enough number 

20 of data preferable for performing pattern recognition at high 
accuracy, for deciding whether the note array (pitch array) in 
the performance data coincides with the pitch array in the data 
stored in the buffer. The buffer can store, for example, 
maximum five data for depressed key data, and when five data 

25 for depressed key data are stored therein, the judgment in step 
S467 becomes affirmative. 

It is desired that the buffer be constructed like a ring 
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buffer, in which the data input later is sequentially 
overwritten on the oldest data. In other words , data relating 
to five notes newly played by key depression, after a note has 
been played at a different pitch from that of the instructed 
5 note, is stored in the buffer. 

If the judgment in step S467 is affirmative, process 
proceeds to step S468 . When a predetermined number of depressed 
key data is not stored, the judgment in step S467 is negative, 
and in this case, process returns to main routine, to start the 
10 processing from step S460 again, when the key event is generated 
next . 

In step S468, pattern recognition, that is, pattern 
comparison between the stored data in the buffer and the 
performance data is performed. The performance data, which is 

15 an object of the pattern recognition, is the data subsequent 
to the current data indicated by the key depression instruction 
pointer. In step S469, it is judged whether the patterns are 
coincide with each other. If patterns are matched, process 
proceeds to step S470, to update the key depression instruction 

20 pointer. Here, the key depression instruction pointer is 
advanced to the position next to the area where the pattern 
matching has been successful. By updating the key depression 
instruction pointer, the key depression instruction mark is 
scrolled. 

25 In step S471, the evaluation relating to the key 

depression tone before the area where the pattern matching has 
been successful is stored in the evaluation storage area. Since 
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there is no key depression tone in the area before success of 
pattern match, low evaluation is written as being not played. 
In step S472, the pitch and the key-on timing are evaluated, 
with respect to the respective depressed key data in the stored 
5 data in the buffer. In step S473, the stored data in the buffer 
and the evaluation results in step S472 are stored in the 
evaluation storage area. In step S4 74, the buffer is cleared. 
When the buffer is cleared, the displayed image for the key 
depression instruction is updated at the same time and scrolled 

10 so as to give the key depression instruction of the current note 
indicated by the pointer. In the case of music with 
accompaniment, the pointer for reproducing the accompaniment 
is also shifted to the corresponding position. 

Fig. 3 0 is a diagram showing the structural example of 

15 the evaluation storage area. In this figure, the pitch of a 
note corresponds with a depressed key and the key-on timing are 
stored as the performance data for each pointer number. In this 
figure, the number indicating the pitch is represented by note 
number. In the same pointer number, there are described the 

20 actual key-on timing and the evaluation thereof, that is, the 
grade (S, A, ...) based on the deviation between the key-on timing 
in the performance data and the actual key-on timing. As the 
performance data for the evaluation, not only the key-on timing 
but also the key-on length and velocity may be included, but 

25 for the brevity of explanation, only the key-on timing is stored 
in the evaluation storage area herein. 

Fig. 31 is a diagram showing the structural example of 
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the depressed key data storage area, that is, the buffer. The 
pitch of the key (note number) actually depressed and the key-on 
timing are stored in the buffer. In this figure, only three 
data are stored, but it is set such that the buffer can store 
5 up to five data. However, the size of the buffer is not limited 
thereto. The data accumulation may be set so as to follow the 
performance, by detecting the music part being played at present 
by pattern recognition. 

Fig. 3 2 is a main functional block diagram showing the 

10 process of the depressed key data. In this figure, a key 
depression detection section 60 detects depressed key data 
input from the detection circuit 23, and inputs this data 
representing depressed key tone to a pitch judgment section 61. 
The pitch judgment section 61 compares the pitch as a note number 

15 in the performance data input according to the current pointer 
from a performance data storage 62 with the pitch as a key number 
of the depressed key tone. The pitch judgment section 61 
outputs an agreement signal when the pitches agree with each 
other, or outputs disagreement signal when the pitches do not 

20 agree with each other. 

A switch 63 is changed over to a key depression evaluation 
section 64 while the agreement signal is output, and changed 
over to a depressed key data storage area (buffer) 65 in response 
to the disagreement signal. Once the switch 63 is changed over 

25 to the buffer 65, the switch 63 is kept in the state changed 
over to the buffer 65, until a matching signal is input from 
a pattern detection section 66. The key depression data is 
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input from the key depression detection section 60 to the 
section to which the switch 63 is changed over. 

The performance data (pitch, key-on timing and the like) 
read from the performance data storage 62 is stored in an 
evaluation storage 67 for evaluation of the key depression. The 
key depression evaluation section 64 compares the stored data 
in the evaluation storage 67 with the key depression data, to 
give predetermined evaluation based on the difference between 
these data. The evaluation is stored in the evaluation storage 
67. When evaluation of the key depression data has finished, 
scroll of the key depression instruction is instructed to a key 
depression instruction section 68. 

The depressed key data storage area 65 stores the key 
depression data in which the key number, that is, a 
representative of the pitch does not agree with the stored data. 
The key depression data stored in the depressed key data storage 
area 65 is input to the pattern detection section 66. The 
pattern detection section 66 searches the performance data 
before the current pointer, of the data in the performance data 
storage62, to detect a pattern (note array) in the performance 
data which agrees with the depressed key data stored in the 
depressed key data storage area 65. When a pattern coincide 
with the depressed key data is detected, a matching signal is 
output, a switch 69 is changed over to the key depression 
evaluation section 64, the depressed key data stored in the 
depressed key data storage 65 is input to the key depression 
evaluation section 64 and evaluated, and the evaluation result 
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is described in the evaluation storage 67. The depressed key 
data storage 65 is cleared by the matching signal, and the switch 
63 is changed over to the key depression evaluation section 64. 

According to this embodiment, when performance is not 
conducted as instructed by the key depression instruction, the 
depressed key data thereafter is stored in the buffer. By 
performing pattern comparison between the stored data and the 
performance data, a coincident portion between the current key 
depression position and the performance data can be detected, 
thereby a pointer for reading the data for the key depression 
instruction can be updated. Moreover, evaluation relating to 
the stored data can be obtained. 

Fig. 3 3 is a main flowchart showing another embodiment 
of the invention. Steps S410 to S416 are processed as in steps 
S301, S302, and step S304 to S307 in Fig. 22. When there is 
performance data to be read, the judgment in step S416 is 
negative, and process proceeds to step S417. When there is no 
performance data, the judgment in step S416 is affirmative, and 
process proceeds to step S418. In step S417, it is judged 
whether the lesson is to be interrupted. If there is no lesson 
interrupting instruction from the player during predetermined 
waiting time, process proceeds to step S413 to continue the key 
depression instruction. If there is a lesson interrupting 
instruction, process proceeds to step S418. 

In step S418, the evaluation of performance stored in a 
buffer (described later) and the key-on timing are transcribed 
in an evaluation storage area similar to that shown in Fig. 30. 
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This buffer is different from the buffer, that is, the depressed 
key data storage 65 in the previous embodiment, which stores 
the performance result after a miss-touch, in that the 
performance results from the beginning are stored therein. 
5 In step S419, result judgment is conducted based on the 

evaluation. In step S420, it is judged whether the performance 
is acceptable, as a result of the result judgment. If the 
performance is acceptable, the processing finishes, and if the 
performance is not acceptable, process proceeds to step S412. 

10 Fig. 34 is a flowchart showing the details of input 

processing (stepS415) of the performance result . InstepS150, 
it is judged whether key depression is performed. If key 
depression is detected, process proceeds to step S151, to store 
the key depression results (key number and key-on timing) in 

15 a depressed key data storage area. In step S152, it is judged 
whether the pitch in the performance data for the key depression 
instruction indicated by a key depression instruction pointer 
is identical with the pitch in the depressed key data, that is, 
the depressed key number. If these are identical, process 

20 proceeds to step S153 , to advance the key depression instruction 
pointer forward. Since the key depression instruction pointer 
is updated, the key depression instruction mark is scrolled. 

When the pitch in the performance data for the key 
depression instruction and the pitch of the key depression data 

25 are not identical, process proceeds to step S154. In step S154, 
it is judged whether the number of unevaluated notes in the 
buffer is consecutive for a predetermined number. The 



predetermined number is a number suitable for pattern 
recognition, and for example, is set to five. If the number 
of unevaluated notes is consecutive for the predetermined 
number (for example, 5), process proceeds to step S155, to 
5 perform pattern recognition where a judgment whether the stored 
data array in the buffer is identical with the performance data 
array is performed. The performance data, which is the object 
of pattern recognition, is not the data subsequent to the 
current key depression instruction pointer, but all data of the 

10 selected music. In this manner, not only when the player has 
a miss-touch, but also when the player returns to and plays an 
optional music part, the music part can be detected by pattern 
recognition, to give the key depression instruction. 

In step S156, it is judged whether the identical pattern 

15 is detected. If the identical pattern is detected, process 
proceeds to step S157 to update the key depression instruction 
pointer. In other words, the key depression instruction 
pointer is advanced to the position subsequent to the area where 
the pattern or the data array is identical. Since the key 

20 depression instruction pointer is updated, the key depression 
instruction mark is scrolled. 

In step SI 58, the unevaluated key depression data in the 
buffer are evaluated, and the evaluation thereof is stored in 
an evaluation storage area in the buffer. The key-on timing 

25 of the instructed note is compared with the input key-on timing 
and the evaluation is determined based on the difference between 
them. 



Fig. 35 is a diagram showing the structural example of 
the depressed key data storage (buffer) according to this 
embodiment. The pitch of sound corresponding to the key 
actually played and the key-on timing are stored in the 
depressed key data storage from the beginning of the lesson. 
Different from the buffer in Fig. 31 , the evaluation storage 
area is provided in this buffer. 

Fig. 36 is a main functional block diagram showing 
processing of depressed key data according to the embodiment 
of Fig . 34 . The key depression data detected by a key depression 
detection section 71 is stored in the depressed key data storage 
72. Of the depressed key data, the key number is input to a 
pitch judgment section 7 3 and the key-on timing is input to a 
key depression evaluation section 74. The key depression 
evaluation section 74 evaluates the depressed key data judged 
that the pitches coincide with each other by the pitch judgment 
section 73. The evaluation is stored in the depressed key data 
storage 72. 

When the pitch judgment section 73 judges that the pitch 
in the depressed key data does not coincide with the instructed 
pitch, a plurality of predetermined number of newest depressed 
key data stored in the depressed key data storage 72 and the 
performance data stored in a performance data storage 7 5 are 
input to a pattern detection section 76, and the performance 
data row or array whose pattern coincides with that of the 
depressed key data is detected. Coincidence of the pattern is 
realized when at least the arrays of the pitch coincides with 
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each other. In addition to the pitch, coincidence of time data 
between notes based on the key-on timing may be detected. When 
the coincidence pattern is detected, a pointer update section 
77 shifts the pointer to the data subsequent to the pattern 
coincidenced performance data. According to this pointer, the 
performance data is read out from the performance data storage 
75 to a key depression instruction display 78. 

In the respective embodiments described above, it is 
assumed that pattern coincidence between the depressed key data 
and the performance data is realized when the respective pitch 
arrays coincide with each other. However, the pattern 
recognition method is not limited thereto, and coincident of 
performance or key depression timing may be detected, in 
addition to the coincident of pitches. 

Pattern coincident does not mean complete coincident 
between the depressed key data and the performance data. Even 
if there is a slight difference in the key-on timing or a 
miss-touched key, when the difference or lack is of a degree 
that a plurality of depressed key data can be judged to have 
a similar pattern as a whole, it can be considered that these 
coincide with each other in view of the coincident of the array, 
that is, the patterns agree with each other. 

Fig. 3 7 is a diagram showing an another example of the 
key depression instruction displayed on the display 14 . In this 
figure, a keyboard figure K is displayed in the lower part of 
the screen. In order to make it easy to see to which key on 
the keyboard the key depression instruction corresponds, the 
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keyboard figure K may be displayed not only on the lower part 
of the displayed image but also on the upper part of the displayed 
image. Which key on the keyboard 21 is to be played is described 
by a key depression instruction mark M above the keyboard figure 
5 K. The key depression instruction mark M is displayed at a key 
position corresponding to the key to be played, with a length 
which allows the player to understand the length of the note, 
for example, with respect to notes for two bars. For example, 
a quarter note is expressed by a length twice the length of an 
10 eighth note, and a mark M indicating a white key to be played 
is displayed wider than a mark M indicating a black key to be 
played. 

On the display, the vertical direction indicates a time 
axis, and the key depression instruction mark M is scrolled 

15 downward with the progress of the musical performance. The key 
corresponding to the key depression instruction mark M closer 
to the lower keyboard figure K indicates a key to be played at 
the earlier timing . The key corresponding to the key depression 
instruction mark M whose lower end abuts against the upper edge 

20 of the keyboard figure K is the key that should be being played 
at present. 

With respect to the key depression instruction mark M, 
an exclamation mark (hereinafter referred to as an "advice 
icon") EM is displayed if necessary. This advice icon EM is 
25 indication means for referencing a performance technique that 
cannot be expressed only by the key depression instruction mark 
M and matters to be noted (hereinafter referred to as "advice 



data"). The player can access an information indicating the 
performance technique such as fingering, by indicating the 
advice icon EM by the mouse 13 during the performance. 

Fig. 38 is a flowchart showing a main process for 
5 displaying key depression instruction where the advice icon is 
included. In the following description, the descriptions for 
the step S501 through S506 are omitted, as they are similar to 
the steps SI, S2 and steps S4 through S7 of Fig. 4. 

In step S507, it is judged whether the advice access 

10 instruction has been input, that is, the advice icon EM has been 
indicated. When it is judged that the advice access instruction 
has been input, process proceeds to step S508, to display the 
advice information corresponding to the advice icon EM on the 
display 14. The access page for the advice information is not 

15 only the one in the memory (hard disk, ROM and the like) of the 
PC body 11, but may be obtained by accessing a website on the 
Internet . 

In step S509, it is judged whether the advice information 
display is to be finished. The information display end icon 

20 is displayed on the advice information display screen, and when 
this information display end icon is operated, the judgment in 
step S509 becomes affirmative. 

In step S510, it is judged whether the predetermined 
lesson has been finished. The entire music may be the training 

25 object, but a music part such as bars may be designated 
beforehand, and this part may be the playing object. When the 
part corresponding to all the designated performance data has 
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been played, the judgment in step S510 becomes affirmative, and 
process proceeds to step S511. When the lesson has not been 
finished, process proceeds from step S510 to step S504. 

In step S511, it is judged whether the evaluation of the 
5 performance result read in step S506 has reached the acceptable 
standard. In step S512, it is judged whether the lesson is to 
be continued. If there is no instruction to continue the lesson 
within a predetermined waiting time, the processing finishes. 
If there is an instruction to continue the lesson, process 

10 proceeds to step S503. The instruction to continue the lesson 
can be input from the keyboard 12, or by indicating the continue 
icon that can be displayed on the display. 

Fig. 39 is a diagram showing an example of information 
structure for searching the advice information, wherein the 

15 information is stored in the performance data, corresponding 
to the advice icon EM. In the figure, "timing" indicates time 
from the beginning of the music, and "sign ID" indicates an 
identification code for specifying a note, a sign such as a clef 
and a pedal mark or an area in the displayed musical score. 

20 Since the access page for the advice information can be held 
not only in the memory of the PC body 11, but also in the server 
on the Internet, a file name on the hard disk or an URL of the 
website on the Internet is described as a link address of the 
access page. 

25 In Fig. 40A, the key depression instruction information 

is shown on the score. Dotted line LI indicates the key 
depression timing. That is, by this line Ll, a tone to be played 
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now is indicated. A mark PI indicating a key on the keyboard 
figure K corresponding to this tone is added thereto. The line 
Ll moves to the right on the displayed image , to indicate the 
tone to be played or key to be depressed. The advice icon EM 
5 is shown above such key depression instruction information. 
The advice icon EM is arranged near a pedal mark, indicating 
that the explanation for the pedal mark can be searched as the 
advice information . 

Fig. 4 OB shows one example of explanation searched in 

10 response to the selection of the advice icon EM in Fig. 4 OA. 

Fig. 41A and 4 IB are diagram showing another example of 
advice information displayed on the display 14. In Fig. 41A, 
.the key depression instruction information is shown on the score. 
The advice icon EM is described on the note, and the performance 

15 technique can be searched as the advice information. 

Fig. 4 IB shows the explanation for the advice icon EM in 
Fig. 41A. Here, of the fingering techniques, a technique 
referred to as "finger crossing" is explained. 

Fig. 42 is a main functional block diagram showing the 

20 processing of the key depression tone data according to one 
embodiment of the present invention. When the advice icon EM 
is operated, a reference file detection section 80 refers to 
a file name storage 81 to detect a file name including the page 
which has the advice information corresponding to the operated 

25 icon. The detected file name is provided to a communication 
section 82, and the communication section 82 accesses a hard 
disk 83, being the memory in the PC body 11 or a website on the 
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Internet 84, based on the file name, to obtain the advice 
information. The obtained advice information is displayed on 
the display 14, using the software set in an access page display 
section 85. 

5 When the advice information is displayed, the advice 

information may be displayed instead of the key depression 
instruction display, or the key depression instruction may be 
left as it is, and the advice information may be superimposed 
thereon and displayed. The advice information may contain only 

10 character information, or for example, may contain animation 
images and photographs for explaining fingering. The advice 
icon informing the existence of the advice information includes 
not only a figure, but also a character or a character string. 
As is obvious from the above explanation, according to 

15 the present inventions , the training part can be designated with 
units having different sizes for each skill level, and when a 
predetermined unit in each skill level reaches the acceptable 
standard, training for the upper skill level is performed. 
Therefore, the skill level can be gradually elevated, while 

20 gradually extending the training range. 

The player can have a look at the progressing degree of 
training in the lesson menu. Since the player does not train 
for the units having the same content repetitively, the training 
efficiency is increased, and the player is not bored due to 

25 monotonous training. 

Since the unit can be reset according to the preference, 
so as to enable easy training of the player, the training 
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efficiency can be increased. 

According to the present inventions, the designation of 
the unit can be performed automatically or manually • Therefore , 
not only a standardized performance instruction in which the 
5 progress is determined automatically is possible, but also the 
progress can be advanced or returned according to the preference 
of the player. 

According to the present inventions, the auto-mode and 
the manual mode can be selected on the lesson menu. During the 

10 auto-mode, when the lesson is interrupted, the lesson menu is 
displayed, so that a unit can be designated manually . Therefore, 
even when the training is executed in the auto-mode, and the 
current continuous units have not been finished, the player can 
choose to go ahead or return to the beginning to have another 

15 training. 

According to the present inventions, when predetermined 
units in each skill level have reached the acceptable standard, 
the player proceeds to training in the upper ranking skill level. 
As a result, while gradually extending the training range, the 

20 player can elevate the skill level gradually. 

According to the present inventions, the performance 
instruction and the performance evaluation can be displayed in 
a plurality of display modes. Particularly, in the score view 
mode, a display suitable for a player who can read the musical 

25 score can be made, and in the piano roll view mode, a display 
in which even a player who cannot read the musical score can 
easily understand the performance instruction can be made. By 
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changing over these display modes, players who cannot 
understand the musical score can learn the correspondence 
between the mark and the score, according to their interest. 
In the superimpose mode, by displaying a plurality of display 

5 modes at the same time, the correspondence between the mark and 
the score can be directly learnt. 

The performance evaluation can be also displayed in a 
preferred display mode, which is easy to understand. 

According to the present invention, since musical 

10 performance is evaluated by comparing the information relating 
to the performance timing with the model information, the 
player's ability, which cannot be judged by the evaluation of 
whether the player has played at the instructed pitch, can be 
judged, such as the rhythmic sense and accuracy of understanding 

15 with respect to the music. According to the present invention, 
unification of the whole performance can be judged. Moreover, 
the evaluation can be visually recognized. Particularly, 
according to the invention of claim 6 , evaluation of performance 
can be seen corresponding to the performance instruction 

20 display. 

According to the present invention, since the player's 
ability can be specifically judged, when the training grade is 
elevated, appropriate result evaluation standard can be set for 
each grade, linked with the judgment result. 
25 According to the invention, the progress of the key 

depression instruction is advanced forward when the key is 
depressed as instructed. However, even if the key is not 



depressed as instructed, the current position in the 
performance data is judged by the pattern recognition method 
based on the depressed key number stored in the buffer. 
Performance is resumed from the position subsequent to the 
5 current position. Therefore, even when the current position 
is lost due to a slight miss-touch or expemporization, the 
position can be found immediately to resume the instruction. 

Since the training is not interrupted due to a slight miss 
or expemporization, the player can freely play without being 
10 forced to do faithful performance as instructed by the 
performance instruction at all times. Moreover, flexible 
training is possible without fearing for a slight mistake. 

According to the invention, even when the depressed key 
number is not as instructed, evaluation is possible with respect 
15 to the stored performed data, and hence the player can recognize 
objective evaluation. 

According to the present invention, the player can obtain 
the advice information according to need, when he/she finds the 
advice icon during performance. Therefore, since the player 
20 can advance the training, while understanding an uncertain 
point such as the style of rendition one by one, the player can 
make progress quickly. Explanation of signs on the musical 
score, which a beginner cannot understand, and techniques that 
do not appear on the musical score can be designated as the advice 
25 information, thereby enabling realization of integrated 
performance instruction. Particularly, since the advice 
information can be obtained through the Internet, the data 
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amount stored in a recording medium equipped in the musical 
performance self -training apparatus can be reduced , and enough 
information can be searched. 
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